To achieve visually compelling results, computer graphics applications may render many virtual objects to the screen, with each virtual object preferably having a high level of detail. The application may typically hold multiple representations of those virtual objects in memory, for example, the geometry of an object or a texture mapped on the geometry, each with a set of different levels-of-detail (LODs), such that the application can select the LOD representation which results in the best visual quality. For example, a tree at a great virtual distance does not have to be rendered with high fidelity, so the texture for that tree may have a low resolution. The virtual world the application renders may contain many more virtual objects and representations each with different LODs than can fit into the application's memory.
A streaming engine is an engine that loads representations with different LOD of the virtual objects in a small but fast cache memory from a large but slow storage. This is done once the virtual object is needed for processing or rendering. These representations are called streamable objects, for example, a texture or a geometry model. To allow for more fine-grained streaming, the streamable objects with their LODs are split into subregions, called tiles, which the streaming engine streams. For example, textures are split into small, rectangular regions of pixels.
Streaming tiles from storage into memory may require a considerable amount of time. If the tile is streamed from a spinning disk to memory, the disk heads need to align, data needs to be read, tiles need to be decompressed and copied to GPU texture memory. During this time the user may be faced with a reduced quality of experience as the tile is not available for rendering. For example, the scene may be rendered partially with no textures until the texture becomes available. The scene may be rendered with a tile with lower quality. However, once the requested tile with higher quality becomes available, the user may be presented with a virtual object of which the rendering fidelity suddenly changes. In other words, the user may see additional detail ‘popping in the screen’ once the tile is loaded.
Clearly, it is preferable to have the tiles required to render a scene available into cache memory and, preferably, to have them ready in the cache memory before or at the moment the application uses them for the first time. If the scene can be rendered solely with tiles present in the cache, there will be no sudden popping of additional detail in the rendered results and the user can always enjoy the highest quality of experience.
By predicting the working set of a virtual scene, i.e., the set of tiles to be used to render the scene, the cache can be optimized and the application does not have to wait until the tile is available for rendering. The delay between a tile request and the moment a tile becomes available for the application, can be hidden and the user may never experience the aforementioned popping of detail.
Precisely predicting the working set allows for better use of bandwidth and computational power as streaming of unused tiles is avoided, but it comes at the cost of increased computational complexity. However, most often, highly complex predictors do not have to be run for every render of a virtual scene and low-complexity predictors may return inaccurate results.
To solve these problems, there is a need for improved tile prediction that also allows for better scaling of the computational load of the predictions on the computer system.
The paper ‘Texture Virtualization for Terrain Rendering’ (D. Cornell, April 2012, pp. 1-10) relates to virtual texturing and presents an overview of virtual texturing technology and virtual texturing pipelines. It discusses how a scene can be rendered to a need buffer which stores information on texture tiles in order to determine the visible texture regions that need to be loaded in a streaming cache. The paper discusses various different stand-alone techniques which can be used by a predictor to determine these visible texture regions. The paper implies using a single predictor for determining said visible texture regions, running in a static configuration for each frame. Hence, only stand-alone specialized techniques are presented. These stand-alone techniques are not shown to have any adaptive properties concerning their activation or deactivation, frequency of execution, computational complexity or combination with other techniques.
The bachelor thesis ‘Virtual Texturing’ (A. Neu, April 2010) presents an overview of virtual texturing technology and virtual texturing pipelines. As in the paper of Cornell, the thesis discusses how the scene can be rendered to a need buffer which stores information on texture tiles in order to determine the visible texture regions that need to be loaded in a streaming cache. Various different stand-alone techniques are discussed which can be used by a predictor to determine the visible texture regions. Also here the use of a single predictor for determining the visible texture regions running in a static configuration for each frame, is implied.
In US2008/147971 a predictive model is used to populate a cache in a videogame system. The predictive model takes as an input a sequence of file sections that have been requested for the associated videogame thus far. The predictive model then returns the names or indicators of one or more file sections that will likely be requested in the future along with a probability that those file sections will be requested. This can be used to reduce the load times during the execution of the video game.